A variety of signal processing applications require geometric transformations of image data to correct for distortions introduced into image data by an optical system. Such geometric transformations generally include reordering signal data in space to correct for the distortions. Current solutions for distortion correction utilize serial data processing in software to transform pixel data. Each pixel in an image is modified based on a specific geometric transformation, and stored in an output memory buffer. After all pixels in the image have been processed, the corrected image may be outputted from the memory buffer. This technique creates a sequential bottleneck in image processing at the memory buffer and therefore fails to provide high-speed and real-time performance for image correction.
For example, U.S. Pat. No. 6,538,691, entitled “Software Correction of Image Distortion in Digital Cameras,” provides a method for using a polynomial model for image distortion correction. The computations for transforming pixel positions are performed using a software application, which outputs transformed pixels to a memory buffer with a modified location calculated using a polynomial model. Such software based techniques for processing pixel data involve a relatively high level of computation complexity, and associated expense, to perform the image processing operations.
In Linear Shift Invariant (LSI) upstream processing, chromatic aberration correction (CAC) has traditionally been handled by the design of high-end achromatic lenses. These geometric distortions are many times focus dependent and their correction needs a complex remapping interpolation. A number of software solutions address the simple types of distortions quite successfully. One such technique involves pre-calibrating the color channels for optimal focus, magnification, and shift. However, producing an aberration free image requires taking three separate images.
When the distortion is more complex, varies with focus, and depends on the color plane, hardware complexity necessitates the use of special lenses, which inevitably increases system costs. Y. Takayuki et al., “A Lateral Chromatic Aberration Correction System for Ultrahigh-Definition Color Video Camera,” Sensors, Cameras and Systems for Scientific and Industrial Applications VII: Proceedings of SPIE (Vol. 6068, SPIE Press, 2006) proposes a Lateral Chromatic Aberration Correction (LCAC) system which includes a real time signal processing configuration stored in memory stacks for variable camera focus settings. Other techniques include image warping using cubic splines and finite dimensional linear modeling, as proposed in Boult and Wolberg “Correcting Chromatic Aberrations Using Image Warping,” IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Proceedings CVPR 1992 (IEEE Computer Society, 2008) and Cai et al. “Color Correction for Digital Images based on the Finite Dimensional Linear-Model,” International Conference on Computer Science an Software Engineering (IEEE Computer Society, 2008) respectively. However, in practice, each of these have severe realization complexities. The difficulty in mapping these complex methods to dedicated hardware limits their use.
Weng et al. “Camera Calibration with Distortion Models and Accuracy Evaluation,” IEEE Transactions on Pattern Analysis and Machine Intelligence (1992) provides an overview of Geometric Distortion Correction (GDC) methods, as well as an evaluation of the error induced by each of the interpolation methods. A normalized stereo calibration error model for compensating the geometric artifacts is introduced. Other linear and nonlinear methods have been developed to tackle distortion correction and include improvements over the basic methods described in Weng. However, as discussed above, none of these approaches consider a hardware mapping cost associated with the corresponding methods.
Besides the CAC and GDC approaches discussed above, there have also been design attempts that involve complexity and/or accuracy tradeoffs. Software methods for GDC include Heikkila and Silven, “A Four-Step Camera Calibration Procedure with Implicit Image Correction,” CVPR '97 Proceedings of the 1997 Conference on Computer Vision and Pattern Recognition (IEEE Computer Society, 1997) and Hartley and Kang, “Parameter Free Radial Distortion Correction with Centre of Distortion Estimation,” ICCV 05: Proceedings of the Tenth IEEE International Conference on Computer Vision (IEEE Computer Society, 2005). Heikkila and Silven propose a radial distortion correction model based on a set of coefficients for camera calibration. Linear and non-linear models are outlined using the Mathwork Matlab's Camera Calibration Toolbox. Hartley and Kang propose a similar distortion correction model. However, they account only for tangential distortion by using a distortion calibration grid captured in several images. U.S. Patent Pub. No. 2006/0280376, entitled “Method for Geometry Distortion Correction,” proposes a technique where an image is predistorted in order to compensate for the distortions which would occur later. This is a pre-processing technique where the image is intentionally distorted before the processing step.